Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 908 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.5476. Min Act: -0.1700

Data Index: 8786828 (Open Web Text)

Max Activating Token Index: 519

Text #1

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.4685. Min Act: -0.1700

Data Index: 1355382 (Open Web Text)

Max Activating Token Index: 82

Text #2

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.4108. Min Act: -0.1700

Data Index: 8017726 (Open Web Text)

Max Activating Token Index: 730

Text #3

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.4148. Min Act: -0.1700

Data Index: 241415 (Open Web Text)

Max Activating Token Index: 861

Text #4

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.3791. Min Act: -0.1700

Data Index: 3977733 (Open Web Text)

Max Activating Token Index: 379

Text #5

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.3422. Min Act: -0.1700

Data Index: 2934918 (Open Web Text)

Max Activating Token Index: 623

Text #6

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.3210. Min Act: -0.1700

Data Index: 8162373 (Open Web Text)

Max Activating Token Index: 846

Text #7

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.3466. Min Act: -0.1700

Data Index: 650512 (Open Web Text)

Max Activating Token Index: 926

Text #8

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.3275. Min Act: -0.1700

Data Index: 6680242 (Open Web Text)

Max Activating Token Index: 725

Text #9

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.3266. Min Act: -0.1700

Data Index: 8560282 (Open Web Text)

Max Activating Token Index: 983

Text #10

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.3169. Min Act: -0.1700

Data Index: 1355389 (Open Web Text)

Max Activating Token Index: 271

Text #11

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.3107. Min Act: -0.1700

Data Index: 441918 (Open Web Text)

Max Activating Token Index: 831

Text #12

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.3168. Min Act: -0.1700

Data Index: 2711656 (Open Web Text)

Max Activating Token Index: 445

Text #13

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.3158. Min Act: -0.1700

Data Index: 5506312 (Open Web Text)

Max Activating Token Index: 754

Text #14

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.3314. Min Act: -0.1700

Data Index: 2577571 (Open Web Text)

Max Activating Token Index: 896

Text #15

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.2852. Min Act: -0.1700

Data Index: 6824151 (Open Web Text)

Max Activating Token Index: 911

Text #16

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.2980. Min Act: -0.1700

Data Index: 3525758 (Open Web Text)

Max Activating Token Index: 353

Text #17

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.2448. Min Act: -0.1700

Data Index: 3071184 (Open Web Text)

Max Activating Token Index: 996

Text #18

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.2259. Min Act: -0.1700

Data Index: 2066704 (Open Web Text)

Max Activating Token Index: 292

Text #19

Max Range: 4.5476. Min Range: -4.5476

Max Act: 4.2389. Min Act: -0.1700

Data Index: 1804386 (Open Web Text)

Max Activating Token Index: 575