Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1741 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.8431. Min Act: -0.1700

Data Index: 4409639 (Open Web Text)

Max Activating Token Index: 489

Text #1

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.7754. Min Act: -0.1700

Data Index: 1977214 (Open Web Text)

Max Activating Token Index: 848

Text #2

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.7914. Min Act: -0.1700

Data Index: 1261160 (Open Web Text)

Max Activating Token Index: 140

Text #3

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.7729. Min Act: -0.1700

Data Index: 4142331 (Open Web Text)

Max Activating Token Index: 820

Text #4

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.7530. Min Act: -0.1700

Data Index: 3316083 (Open Web Text)

Max Activating Token Index: 118

Text #5

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.7029. Min Act: -0.1700

Data Index: 1301872 (Open Web Text)

Max Activating Token Index: 529

Text #6

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.6037. Min Act: -0.1700

Data Index: 6387026 (Open Web Text)

Max Activating Token Index: 626

Text #7

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.6244. Min Act: -0.1700

Data Index: 1289880 (Open Web Text)

Max Activating Token Index: 225

Text #8

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.6285. Min Act: -0.1700

Data Index: 6723804 (Open Web Text)

Max Activating Token Index: 675

Text #9

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.5857. Min Act: -0.1700

Data Index: 1855859 (Open Web Text)

Max Activating Token Index: 735

Text #10

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.5910. Min Act: -0.1700

Data Index: 6948947 (Open Web Text)

Max Activating Token Index: 1020

Text #11

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.5518. Min Act: -0.1700

Data Index: 520990 (Open Web Text)

Max Activating Token Index: 98

Text #12

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.5981. Min Act: -0.1700

Data Index: 5283031 (Open Web Text)

Max Activating Token Index: 598

Text #13

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.5011. Min Act: -0.1700

Data Index: 4976657 (Open Web Text)

Max Activating Token Index: 749

Text #14

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.5180. Min Act: -0.1700

Data Index: 3422763 (Open Web Text)

Max Activating Token Index: 905

Text #15

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.5025. Min Act: -0.1700

Data Index: 6757658 (Open Web Text)

Max Activating Token Index: 468

Text #16

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.5331. Min Act: -0.1700

Data Index: 7239731 (Open Web Text)

Max Activating Token Index: 282

Text #17

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.4741. Min Act: -0.1700

Data Index: 7483779 (Open Web Text)

Max Activating Token Index: 333

Text #18

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.5089. Min Act: -0.1700

Data Index: 8329242 (Open Web Text)

Max Activating Token Index: 632

Text #19

Max Range: 5.8431. Min Range: -5.8431

Max Act: 5.5142. Min Act: -0.1700

Data Index: 1898535 (Open Web Text)

Max Activating Token Index: 497