Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1102 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.5498. Min Act: -0.1700

Data Index: 6195762 (Open Web Text)

Max Activating Token Index: 987

Text #1

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.4882. Min Act: -0.1699

Data Index: 5072152 (Open Web Text)

Max Activating Token Index: 974

Text #2

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.5246. Min Act: -0.1700

Data Index: 7315659 (Open Web Text)

Max Activating Token Index: 717

Text #3

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.4477. Min Act: -0.1700

Data Index: 3494965 (Open Web Text)

Max Activating Token Index: 995

Text #4

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.3921. Min Act: -0.1700

Data Index: 5743494 (Open Web Text)

Max Activating Token Index: 361

Text #5

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.4047. Min Act: -0.1700

Data Index: 5347189 (Open Web Text)

Max Activating Token Index: 547

Text #6

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.3331. Min Act: -0.1700

Data Index: 2760171 (Open Web Text)

Max Activating Token Index: 413

Text #7

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.3837. Min Act: -0.1700

Data Index: 2721157 (Open Web Text)

Max Activating Token Index: 981

Text #8

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.3871. Min Act: -0.1700

Data Index: 8588529 (Open Web Text)

Max Activating Token Index: 590

Text #9

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.2779. Min Act: -0.1700

Data Index: 2585112 (Open Web Text)

Max Activating Token Index: 682

Text #10

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.2529. Min Act: -0.1700

Data Index: 6519784 (Open Web Text)

Max Activating Token Index: 355

Text #11

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.2801. Min Act: -0.1700

Data Index: 1623240 (Open Web Text)

Max Activating Token Index: 900

Text #12

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.2251. Min Act: -0.1700

Data Index: 383852 (Open Web Text)

Max Activating Token Index: 890

Text #13

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.2177. Min Act: -0.1700

Data Index: 4737860 (Open Web Text)

Max Activating Token Index: 263

Text #14

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.1601. Min Act: -0.1700

Data Index: 1216151 (Open Web Text)

Max Activating Token Index: 546

Text #15

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.1488. Min Act: -0.1700

Data Index: 5322789 (Open Web Text)

Max Activating Token Index: 717

Text #16

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.1422. Min Act: -0.1700

Data Index: 8271926 (Open Web Text)

Max Activating Token Index: 753

Text #17

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.1838. Min Act: -0.1700

Data Index: 7004121 (Open Web Text)

Max Activating Token Index: 314

Text #18

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.1167. Min Act: -0.1700

Data Index: 4003738 (Open Web Text)

Max Activating Token Index: 227

Text #19

Max Range: 6.5498. Min Range: -6.5498

Max Act: 6.1985. Min Act: -0.1700

Data Index: 5976622 (Open Web Text)

Max Activating Token Index: 476