Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2197 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.5632. Min Act: -0.1700

Data Index: 1937846 (Open Web Text)

Max Activating Token Index: 531

Text #1

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.4065. Min Act: -0.1700

Data Index: 3859859 (Open Web Text)

Max Activating Token Index: 277

Text #2

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.3658. Min Act: -0.1700

Data Index: 4750839 (Open Web Text)

Max Activating Token Index: 30

Text #3

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.3274. Min Act: -0.1700

Data Index: 1464860 (Open Web Text)

Max Activating Token Index: 550

Text #4

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.3151. Min Act: -0.1700

Data Index: 310144 (Open Web Text)

Max Activating Token Index: 843

Text #5

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.3226. Min Act: -0.1700

Data Index: 1699639 (Open Web Text)

Max Activating Token Index: 73

Text #6

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.3127. Min Act: -0.1700

Data Index: 3378739 (Open Web Text)

Max Activating Token Index: 242

Text #7

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.2369. Min Act: -0.1700

Data Index: 3885995 (Open Web Text)

Max Activating Token Index: 1004

Text #8

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.1919. Min Act: -0.1700

Data Index: 3260194 (Open Web Text)

Max Activating Token Index: 621

Text #9

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.2105. Min Act: -0.1700

Data Index: 6397033 (Open Web Text)

Max Activating Token Index: 825

Text #10

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.2121. Min Act: -0.1700

Data Index: 2778031 (Open Web Text)

Max Activating Token Index: 447

Text #11

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.1975. Min Act: -0.1700

Data Index: 5132178 (Open Web Text)

Max Activating Token Index: 129

Text #12

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.1660. Min Act: -0.1700

Data Index: 6304791 (Open Web Text)

Max Activating Token Index: 848

Text #13

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.1626. Min Act: -0.1700

Data Index: 8014433 (Open Web Text)

Max Activating Token Index: 143

Text #14

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.1584. Min Act: -0.1700

Data Index: 2639407 (Open Web Text)

Max Activating Token Index: 926

Text #15

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.1567. Min Act: -0.1700

Data Index: 3087284 (Open Web Text)

Max Activating Token Index: 48

Text #16

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.1280. Min Act: -0.1700

Data Index: 5119818 (Open Web Text)

Max Activating Token Index: 859

Text #17

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.1575. Min Act: -0.1700

Data Index: 311339 (Open Web Text)

Max Activating Token Index: 424

Text #18

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.1327. Min Act: -0.1700

Data Index: 2190615 (Open Web Text)

Max Activating Token Index: 460

Text #19

Max Range: 4.5632. Min Range: -4.5632

Max Act: 4.1194. Min Act: -0.1700

Data Index: 4158701 (Open Web Text)

Max Activating Token Index: 437