Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1306 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.4622. Min Act: -0.1700

Data Index: 1789335 (Open Web Text)

Max Activating Token Index: 334

Text #1

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.4278. Min Act: -0.1700

Data Index: 2991413 (Open Web Text)

Max Activating Token Index: 564

Text #2

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.3250. Min Act: -0.1700

Data Index: 3441363 (Open Web Text)

Max Activating Token Index: 888

Text #3

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.3952. Min Act: -0.1700

Data Index: 8728364 (Open Web Text)

Max Activating Token Index: 325

Text #4

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.3915. Min Act: -0.1700

Data Index: 7137333 (Open Web Text)

Max Activating Token Index: 1019

Text #5

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.3760. Min Act: -0.1700

Data Index: 7928108 (Open Web Text)

Max Activating Token Index: 645

Text #6

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.3224. Min Act: -0.1700

Data Index: 6446517 (Open Web Text)

Max Activating Token Index: 378

Text #7

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.3313. Min Act: -0.1700

Data Index: 1641019 (Open Web Text)

Max Activating Token Index: 139

Text #8

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.2940. Min Act: -0.1700

Data Index: 8563077 (Open Web Text)

Max Activating Token Index: 983

Text #9

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.2791. Min Act: -0.1700

Data Index: 5622794 (Open Web Text)

Max Activating Token Index: 885

Text #10

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.2140. Min Act: -0.1700

Data Index: 2090285 (Open Web Text)

Max Activating Token Index: 777

Text #11

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.2775. Min Act: -0.1700

Data Index: 3106948 (Open Web Text)

Max Activating Token Index: 877

Text #12

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.2743. Min Act: -0.1700

Data Index: 7478132 (Open Web Text)

Max Activating Token Index: 417

Text #13

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.2443. Min Act: -0.1700

Data Index: 4742887 (Open Web Text)

Max Activating Token Index: 378

Text #14

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.2133. Min Act: -0.1700

Data Index: 7989320 (Open Web Text)

Max Activating Token Index: 352

Text #15

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.2310. Min Act: -0.1700

Data Index: 4619031 (Open Web Text)

Max Activating Token Index: 335

Text #16

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.2646. Min Act: -0.1700

Data Index: 301689 (Open Web Text)

Max Activating Token Index: 257

Text #17

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.1730. Min Act: -0.1700

Data Index: 8499830 (Open Web Text)

Max Activating Token Index: 953

Text #18

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.2443. Min Act: -0.1700

Data Index: 7051482 (Open Web Text)

Max Activating Token Index: 365

Text #19

Max Range: 4.4622. Min Range: -4.4622

Max Act: 4.2530. Min Act: -0.1700

Data Index: 1791909 (Open Web Text)

Max Activating Token Index: 380