Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 839 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.7349. Min Act: -0.0000

Data Index: 550870 (The Pile)

Max Activating Token Index: 520

Text #1

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.6653. Min Act: -0.0000

Data Index: 1622711 (The Pile)

Max Activating Token Index: 401

Text #2

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.6435. Min Act: -0.0000

Data Index: 677928 (The Pile)

Max Activating Token Index: 96

Text #3

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.6353. Min Act: -0.0000

Data Index: 420511 (The Pile)

Max Activating Token Index: 460

Text #4

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.5705. Min Act: -0.0000

Data Index: 1475238 (The Pile)

Max Activating Token Index: 324

Text #5

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.5893. Min Act: -0.0000

Data Index: 1811630 (The Pile)

Max Activating Token Index: 156

Text #6

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.6202. Min Act: -0.0000

Data Index: 1412381 (The Pile)

Max Activating Token Index: 721

Text #7

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.5618. Min Act: -0.0000

Data Index: 396172 (The Pile)

Max Activating Token Index: 716

Text #8

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.5262. Min Act: -0.0000

Data Index: 383976 (The Pile)

Max Activating Token Index: 109

Text #9

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.5124. Min Act: -0.0000

Data Index: 97587 (The Pile)

Max Activating Token Index: 626

Text #10

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.5112. Min Act: -0.0000

Data Index: 1656286 (The Pile)

Max Activating Token Index: 903

Text #11

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.5381. Min Act: -0.0000

Data Index: 443543 (The Pile)

Max Activating Token Index: 274

Text #12

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.5217. Min Act: -0.0000

Data Index: 457051 (The Pile)

Max Activating Token Index: 668

Text #13

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.4641. Min Act: -0.0000

Data Index: 1518769 (The Pile)

Max Activating Token Index: 911

Text #14

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.5263. Min Act: -0.0000

Data Index: 972652 (The Pile)

Max Activating Token Index: 597

Text #15

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.4768. Min Act: -0.0000

Data Index: 242166 (The Pile)

Max Activating Token Index: 684

Text #16

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.4717. Min Act: -0.0000

Data Index: 1986442 (The Pile)

Max Activating Token Index: 110

Text #17

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.4639. Min Act: -0.0000

Data Index: 1675455 (The Pile)

Max Activating Token Index: 812

Text #18

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.4624. Min Act: -0.0000

Data Index: 1080412 (The Pile)

Max Activating Token Index: 552

Text #19

Max Range: 0.7349. Min Range: -0.7349

Max Act: 0.4812. Min Act: -0.0000

Data Index: 1530549 (The Pile)

Max Activating Token Index: 774